Feature implementation from commits caf17ec..c9e752c#2
Feature implementation from commits caf17ec..c9e752c#2yashuatla wants to merge 15 commits intofeature-base-2from
Conversation
add support for bitnet2b_2501 model
…microsoft#171) * update readme and setup file for new model. * update model file name --------- Co-authored-by: Yan Xia <yanxia@microsoft.com>
add two FAQs for windows build requestions.
add third-party demo
refine readme
Fix model architecture name
…crosoft#204) * Update CMakeLists.txt I added a CMake option to compile the Llama.cpp server. This update allows us to easily build and deploy the server using BitNet * Create run_inference_server.py same as run_inference, but for use with llama.cpp's built in server, for some extra comfort In particular: - The build directory is determined based on whether the system is running on Windows or not. - A list of arguments (`--model`, `-m` etc.) is created. - The main argument list is parsed and passed to the `subprocess.run()` method to execute the system command.
…d/aarch64 (microsoft#242) GCC does not recognize Clang-specific warning flags like -Wunreachable-code-break and -Wunreachable-code-return, which are passed by upstream submodules (e.g., ggml). This patch forces CMake to use Clang via command-line arguments, avoiding the need to patch nested submodules. This resolves compiler errors without modifying submodule source code.
![codeowlai[bot]](https://avatars.githubusercontent.com/in/1259765?s=60&v=4){kind=small}
| exit(0) | ||
| logging.info("Compiling the code using CMake.") | ||
| run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files") | ||
| run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files") |
There was a problem hiding this comment.
🐛 Correctness Issue
Hardcoded compiler breaks cross-platform compatibility.
Hardcoding clang/clang++ will cause build failures on systems where these compilers aren't available or where different compilers are required.
Current Code (Diff):
- run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files")
+ run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files")📝 Committable suggestion
‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀
| run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files") | |
| run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files") |
| ) -> Tuple[np.ndarray, np.ndarray]: | ||
| M, K = w.shape | ||
|
|
||
| cf=configparser.ConfigParser() |
There was a problem hiding this comment.
🐛 Correctness Issue
Hardcoded path with no error handling.
The code reads a configuration file from a hardcoded path without checking if the file exists, which will cause runtime errors if the file is missing.
Current Code (Diff):
- cf=configparser.ConfigParser()
- cf.read("./build/kcfg.ini")
+ cf = configparser.ConfigParser()
+ try:
+ if not os.path.exists("./build/kcfg.ini"):
+ raise FileNotFoundError("Configuration file not found")
+ cf.read("./build/kcfg.ini")
+ if len(cf.sections()) == 0:
+ raise ValueError("Configuration file is empty or invalid")
+ except (FileNotFoundError, ValueError) as e:
+ logger.error(f"Error loading configuration: {e}")
+ raise📝 Committable suggestion
‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀
| cf=configparser.ConfigParser() | |
| cf = configparser.ConfigParser() | |
| try: | |
| if not os.path.exists("./build/kcfg.ini"): | |
| raise FileNotFoundError("Configuration file not found") | |
| cf.read("./build/kcfg.ini") | |
| if len(cf.sections()) == 0: | |
| raise ValueError("Configuration file is empty or invalid") | |
| except (FileNotFoundError, ValueError) as e: | |
| logger.error(f"Error loading configuration: {e}") | |
| raise |
|
|
||
| return w | ||
|
|
||
| def transform_to_i2(x : NDArray): |
There was a problem hiding this comment.
🐛 Correctness Issue
Potential division by zero.
The transform_to_i2 function doesn't properly handle the case where all values in tile_x are zero, which could lead to unexpected behavior.
Current Code (Diff):
-def transform_to_i2(x : NDArray):
- x_num = np.prod(x.shape)
- tile_x = np.reshape(x, x_num)
- scale = 1
- for i in range(x_num):
- if tile_x[i] != 0:
- scale = tile_x[i]
- break
- tile_x = np.divide(tile_x, scale)
- tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8)
- ans = np.reshape(tile_x, x.shape)
- return ans, scale
+def transform_to_i2(x : NDArray):
+ x_num = np.prod(x.shape)
+ tile_x = np.reshape(x, x_num)
+ scale = 1
+ found_nonzero = False
+ for i in range(x_num):
+ if tile_x[i] != 0:
+ scale = tile_x[i]
+ found_nonzero = True
+ break
+
+ if not found_nonzero:
+ # Handle the all-zeros case
+ return np.zeros(x.shape, dtype=np.uint8), scale
+
+ tile_x = np.divide(tile_x, scale)
+ tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8)
+ ans = np.reshape(tile_x, x.shape)
+ return ans, scale📝 Committable suggestion
‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀
| def transform_to_i2(x : NDArray): | |
| def transform_to_i2(x : NDArray): | |
| x_num = np.prod(x.shape) | |
| tile_x = np.reshape(x, x_num) | |
| scale = 1 | |
| found_nonzero = False | |
| for i in range(x_num): | |
| if tile_x[i] != 0: | |
| scale = tile_x[i] | |
| found_nonzero = True | |
| break | |
| if not found_nonzero: | |
| # Handle the all-zeros case | |
| return np.zeros(x.shape, dtype=np.uint8), scale | |
| tile_x = np.divide(tile_x, scale) | |
| tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8) | |
| ans = np.reshape(tile_x, x.shape) | |
| return ans, scale |
5 participants
This PR contains changes from a range of commits from the original repository.
Commit Range:
caf17ec..c9e752cFiles Changed: 7 (4 programming files)
Programming Ratio: 57.1%
Commits included:
... and 5 more commits